Psychological assessment of AI-based decision support systems: tool development and expected benefits

This study aimed to develop an evaluation tool that assesses the use of AI-based decision support systems (DSSs) in professional practice from a human-centered perspective. Following International Organization for Standardization, this perspective aims ensure interactive technologies improves users' psychological load experience and behavior, e.g., form reduced stress or increased performance. Concomitantly, attempts proactively prevent detect correct potential negative effects these on user load, such as impaired satisfaction engagement, early possible. Based perspective, we developed validated questionnaire instrument, Psychological Assessment DSSs (PAAI), user-centered practice. In particular, instrument considers central design characteristics corresponding work situation, which have significant impact load. The was tested two independent studies. Study 1, N = 223 individuals were recruited. results item scale analyses exploratory factor analysis, newly refined, final version using confirmatory analysis. Findings showed acceptable-to-good fit indices, confirming factorial validity PAAI . confirmed second study, had 471 participants. Again, CFA yielded indices. further convergent criterion analyses.